AMENDMENT TO THE CLAIMS 



Please amend the presently pending claims as follows: 

1. (Currently Amended) A computer-implemented method for populating an electronic 
form from an electronic image, the method comprising: 

(a) identifying a size, orientation and position of a first object having any arbitrary 

orientation within the electronic image; 

(b) identifying information elements from pixels within the electronic image that 

correspond to the first object , including identifying text blocks within the first 
object using optical character recognition : 

(c) displaying simultaneously to a user fields of the electronic form in a form data area 

and the identified information elements to a user text blocks in an object data area 
that is outside of the form data area, which corresponds to the first object, through 
a graphical user interface , wherein the text blocks are selectable by the user within 
the object data area through the graphical user interface for insertion into 
respective fields of the electronic form ; 

(d) parsing the information elements into tagged groups of different information types; 

and 

(e) populating the fields of the electronic form with the tagged groups to produce a 

populated form and allowing the user to edit the populated fields through the 
graphical user interface. 

2. (Original) The method of claim 1 wherein (a) comprises identifying a size, orientation and 
position of the first object among a plurality of objects within the electronic image. 

3. (Original) The method of claim 1 wherein (a) comprises: 

classifying each pixel within the image to produce pixel classification data; 
defining an image function to process the pixel classification data; 
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dividing the image into sub-images based on disparities in the image function; and 
processing the sub-images to determine a size, an orientation and a position for each of 
the objects, including the first object. 

4. (Original) The method of claim 3 and further comprising repeating the classifying pixels, 
defining an image function, and dividing of the image until the image contains a single object or 
the image can no longer be divided. 

5. (Original) The method of claim 3 wherein classifying comprises classifying each pixel as 
one of a data pixel or a background pixel. 

6. (Original) The method of claim 5 and further comprising: 

defining the image function as a sum of data pixels in a certain direction; 
calculating the image function in a first direction to generate a first data set; 
calculating the image function in a second direction to generate a second data set; and 
searching for disparities in the image function in the first direction and the image function in 
the second direction. 

7. (Currently Amended). The method of claim 1 wherein: 

(b) comprises identifying text blocks and two-dimensional locations of the text blocks 

within the first object using optical character recognition; and 

(c) comprises displaying the identified text blocks to the user through the graphical user 

interface simultaneously with th e fi e lds of the electronic form . 

8. (Currently Amended) The method of claim 7 wherein: 

(c) comprises displaying to the user the identified information elements in the object data 
area within an image of the first object to the user through the graphical user 
interface; and 



(d) comprises highlighting the tagged groups of different information types within the 
image of the first object in the object data area with visual indicators that indicate 
the different information types. 



9. (Original) The method of claim 1 wherein: 

(d) comprises receiving the information elements as untagged media from an untagged 

media store, and parsing the untagged media to identify information types of the 
information elements; 

(e) comprises automatically populating at least one of the fields with at least one of the 

information elements based on the information type of that element; and 
(c) comprises displaying the information elements through an object data graphical user 
interface and displaying the populated fields and any unpopulated fields through a 
form graphical user interface. 

10. (Original) The method of claim 9 wherein (c) further comprises employing visual indicator 
in the object data graphical user interface to indicate that an information element is compatible 
with a particular field in the form. 

11. (Original) The method of claim 9 wherein (c) further comprises indicating a status level 
associated with at least one information element in at least one populated field in the form. 

12. (Original) The method of claim 11 wherein (c) further comprises prompting the user to 
verify and/or correct the at least one information element in the at least one populated field based 
on the indicated status level. 

13. (Original) The method of claim 12 wherein (d) comprises updating other fields of the 
populated form based on the user verification and/or correction of the at least one information 
element. 
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14. (0riginal) The method of claim 1 and further comprising: 

(f) writing side information gleaned from edits made to any of the populated fields to a 

side information store; and 

(g) re-parsing the information elements into tagged groups of different information types 

in (d) and re-populating at least one other field in (e) based on the side 
information. 

15. (Original) The method of claim 1 wherein the electronic image comprises a plurality of 
objects and the method further comprises: 

(f) performing (a) through (e) for each object, such that, for each object, fields of a 
corresponding electronic form are at least partially populated with information 
elements obtained from pixels within that object. 

16. (Original) The method of claim 15 wherein the plurality of objects comprises objects of 
different types. 

17. (Currently Amended) A computer-readable medium comprising computer storage media 
and computer-executable instructions , at least some of which are stored on the computer storage 
media and that, when executed by a computer, performs perform a method comprising: 

(a) identifying a size, orientation and position of a first object having any arbitrary 

orientation within an electronic image; 

(b) identifying information elements from pixels within the electronic image that 

correspond to the first object , including identifying text blocks within the first 
object using optical character recognition ; 

(c) displaying simultaneously to a user fields of the electronic form in a form data area 

and the identified information elements to a user text blocks in an object data area 
that is outside of the form data area, which corresponds to the first object, through 
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a graphical user interface , wherein the text blocks are selectable by the user within 
the object data area through the graphical user interface for insertion into 
respective fields of the electronic form ; and 

(d) parsing the information elements into tagged groups of different information types; 

(e) populating the fields of the electronic form with the tagged groups to produce a 

populated electronic form and allowing the user to edit the populated fields 
through the graphical user interface. 

18. (Original) The computer- readable medium of claim 17 wherein the electronic image 
comprises a plurality of objects, including the first object, and the method further comprises: 

(f) performing (a) through (e) for each object, such that, for each object, fields of a 

corresponding electronic form are at least partially populated with information 
elements obtained from pixels within that object. 

19. (Original) The computer-readable medium of claim 18 wherein the plurality of objects 
comprises objects of different types. 

20. (Original) The computer-readable medium of claim 17 wherein (a) comprises: 

classifying each pixel within the image to produce pixel classification data; 
defining an image function to process the pixel classification data; 
dividing the image into sub-images based on disparities in the image function; and 
processing the sub-images to determine a size, an orientation and a position for each of the 
objects, including the first object. 

21. (Original) The computer-readable medium of claim 20 wherein classifying comprises 

classifying each pixel as one of a data pixel or a background pixel. 

22. (Original) The computer-readable medium of claim 21 and further comprising: 
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defining the image function as a sum of data pixels in a certain direction; 
calculating the image function in a first direction to generate a first data set; 
calculating the image function in a second direction to generate a second data set; and 
searching for disparities in the image function in the first direction and the image function in 
the second direction. 

23. (Currently Amended) The computer-readable medium of claim 17 wherein: 

(b) comprises identifying text blocks and two-dimensional locations of the text blocks 

within the first object using optical character recognition ; 

(c) comprises displaying the identified text blocks within an image of the first object to 

the user through the graphical user interface; and 

(d) comprises highlighting the tagged groups of different information types within the 

image of the first object in the object data area with visual indicators that indicate 
the different information types. 

24. (Original) The computer-readable medium of claim 17 wherein: 

(d) comprises receiving the information elements as untagged media from an untagged 

media store, and parsing the untagged media to identify information types of the 
information elements; 

(e) comprises automatically populating at least one of the fields with at least one of the 

information elements based on the information type of that element; and 
(c) comprises displaying the information elements through an object data graphical user 
interface and displaying the populated fields and any unpopulated fields through a 
form graphical user interface. 

25. (Original) The computer-readable medium of claim 24 wherein (c) further comprises 
employing visual indicator in the object data graphical user interface to indicate that an 
information element is compatible with a particular field in the form. 
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26. (Original) The computer-readable medium of claim 24 wherein (c) further comprises 
indicating a status level associated with at least one information element in at least one populated 
field in the form. 

27. (Original) The computer-readable medium of claim 26 wherein (c) further comprises 
prompting the user to verify and/or correct the at least one information element in the at least one 
populated field based on the indicated status level. 

28. (Original) The computer-readable medium of claim 27 wherein (d) comprises updating other 
fields of the populated form based on the user verification and/or correction of the at least one 
information element. 

29. (Original) The computer-readable medium of claim 17 and further comprising: 

(f) writing side information gleaned from edits made by the user in step (e) to a side 

information store; and 

(g) re-parsing the parsing the information elements into tagged groups of different 

information types in (d) and re-populating at least one other field in (e) based on 
the side information. 

30. (Currently Amended) A system for at least partially populating electronic forms, the 
system comprising: 

an object detection and extraction module, which processes pixels in the electronic image 

to identify a size, orientation and position of an object having any arbitrary 

orientation within the electronic image; 
an optical character recognition module, which identifies information elements , including 

text blocks, from pixels within the electronic image that correspond to the first 

object; 
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a graphical user interface, which simultaneously displays to a user fields of the electronic 
form in a form data area and the identified information elements to a user the 
identified text blocks in an object data area that is outside of the form data area 
and corresponds to the first object, and wherein the text blocks are selectable by 
the user within the object data area through the graphical user interface for 
insertion into respective fields of the electronic form ; and 

a parsing module, which parses the information elements into tagged groups of different 
information types and at least partially populates the fields with the tagged groups 
to produce a populated electronic form. 

31. (Original) The system of claim 30 wherein the graphical user interface allows the user to edit 
the populated fields and any unpopulated fields. 

32. (Original) The system of claim 30 wherein the object detection and extraction module 
comprises: 

a data pixel detection module that classifies each pixel of the image and defines an image 
function; 

a segmentation module that is capable of dividing the image into smaller sub-images based 

on disparities in the image function; and 
a single object extraction module that processes the sub-images to detect a number, a size, 

an orientation and a position of objects within the image. 

33. (Original) The system of claim 32 wherein the data pixel detection module classifies each 
pixel as one of a data pixel or a background pixel. 

34. (Previously Presented) The system of claim 33 wherein the data pixel detection module 
defines the image function as a sum of data pixels in a certain direction, calculates the image 
function in a first direction to generate a first data set, calculates the image function in a second 
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direction to generate a second data set, and searches for disparities in the image function in the first 
direction and the image function in the second direction. 

35. (Currently Amended) The system of claim 30 wherein: 

the optical character recognition module comprises means for identifying text blocks and 
two-dimensional locations of the text blocks within the first object using optical 
character recognition ; and 

the graphical user interface comprises means for displaying the identified text blocks 
within an image of the first object to the user through the graphical user interface 
and highlighting the tagged groups of different information types within the image 
of the first object with visual indicators that indicate the different information 
types. 

36. (Previously Presented) The system of claim 30 wherein: 

the parsing module comprises means for receiving the information elements as untagged 
media from an untagged media store, parsing the untagged media to identify 
information types of the information elements, and automatically populating at 
least one of the fields with at least one of the information elements based on the 
information type of that element; and 

the graphical user interface comprises means for displaying the information elements 
through an object data graphical user interface and displaying the populated fields 
and any unpopulated fields through a form graphical user interface. 

37. (Currently Amended) The system of claim 3§ 36 wherein the object data graphical user 
interface comprises a visual indicator to indicate that an information element is compatible with a 
particular field in the form. 

3 8. (Currently Amended) The system of claim 35- 36 wherein the form graphical user 
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interface comprises a status level indicator associated with at least one information element in at 
least one populated field in the form. 

3 9. (Currently Amended) The system of claim 33- 38 wherein the graphical user interface 
comprises means for prompting the user to verify and/or correct at least one information element 
in at least one populated field based on the status level indicator associated with that field. 

40. (Previously Presented) The system of claim 31 wherein the parsing module comprises 
means for updating other fields of the populated form according to edits made by the user to the 
populated fields and the unpopulated fields. 

41. (Previously Presented) A method for populating electronic forms from an electronic image 
having first and second objects of different information types, the method comprising: 

identifying a size, orientation and position of the first and second objects within the 
electronic image and dividing the electronic image into sub-images corresponding 
to pixels in the electronic image associated with the size, orientation and position 
of each object; 

performing optical character recognition on each sub-image to identify untagged 

information elements within the corresponding object; 
for each sub-image, parsing the untagged information elements into tagged information 

elements; 

populating fields in a first electronic form type with the tagged information elements 
identified from the sub-image of the first object to produce a first populated form; 

populating fields in a second electronic form type with the tagged information elements 
identified from the sub-image of the second object to produce a second populated 
form; and 

displaying the first and second populated forms and the untagged information elements to 
a user through a graphical user interface and allowing the user to edit the first and 
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second populated foims through the graphical user interface. 

42. (Currently Amended) The method of claim 40 4J_ wherein: 

the first object comprises a business card and the second object comprises a purchase 
receipt; and 

the first electronic form comprises a contact record of a software address book and the 
second electronic form comprises an electronic financial record of a software 
financial application. 



